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ABSTRACT 

Clinchy and Rosenthal's error classification scaeme 
was applied to test results to determine its ability to differentiate 
the effectiveness of instruction in two elementary schools. 
Mathematics retention tests matching the instructional objectives or 
both schools were constructed to measure the understanding oi 
arithmetic concepts and the ability to perform computations and 
algorithmic operations. Inter-school comparisons were made witn 
respect to the types of errors made: computational, algorithmic, aa d 
omission. Fifth grade students in one school made significantly lewer 
computational errors than in the other school. In the remaining 
grades, there were no significant differences. Additional practice 
exercises were recommended to correct this error. Other students made 
significantly more algorithmic errors and a four-step approach to 
teaching algorithms was outlined to alleviate this weakness. 
Significant inter-school differences in omissions errors were notei 
in grades 5 and 6 — the students concerned would also benefit by a 
■ore systematic approach to algorithm teaching. In sum, the error 
classification scheme can be used to evaluate instructional prograus 
and to suggest instructional improvements. (CP) 
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Abstract 

The study examined the usefulness of Clinchy and Rosenthal's 
error classification scheme for the purpose of using test results to 
improve instructional effectiveness. Students in two instructional programs 
were compared with respect to the types of errors made on a test of mathematics 
retention. Three types of errors were identified: computational, algorithmic, 
and omissions. Differences in the types of errors made by the students in 
the two programs were found. In addition, prescriptions for improving 
the programs based on the nature of the errors can be offered. 
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Using Errors to Improve the Oualitv 
of Instructional Programs 

Evaluation results are used to determine whether to maintain, terminate, 
or modifv instructional programs. The decision to maintain or terminate an 
instructional program is typicallv made on the basis of the relationship 
between the actual performance of a group of students and the '^ideal*' 
performance of students as specified by the program. If the actual perfor- 
mance of students surpasses the ^'icleal'' per f ormance standard, the decision is 
made to maintain the program as is. On the other hand, if the actual 
performance of students is far below the standard speci^^ied bv the program, 
the decision is made to terminate the program. 

The problem occurs when the discrepancy between the actual performance 
of students and the level of performance specified by the program is small, 
but noticeable. Typically, the decision in this situation is to modifv 
the program in an effort to improve its effectiveness. U^ilc a decision 
to modify implies that the program is salvageable, specific ways in which 
the program can be improved are not immediately evident. 

Perhaps one reason for the lack of suggestions for improvement of programs 
comes from the manner in which the initial decisions are made. Since the 
decisions to maintain, terminate or modify programs are made based on the 
size of the discrepancy between actual performance and **ideal'* oerformance, 
the most appropriate measure of actual performance is the number of correct 
items. ^'Thile the number of correct items can be used to judge the extent to 
which students attain or surpass a particular standard, it is of little use 
in determining ways of improving instructional programs. Different infor- 
mation seems necessary in order to accomplish this latter task. 



One such source of information Ls the nature of errors made by the 
students in the program. Since errors typically indicate misunderstandings 
that need to be corrected, the type of error made may provide a clue to 
alterations of instructional programs that may lead to some kind of 
improvement. l^Aiat appears necessary, then, for errors to be useful in this 
regard is some typology of errors. Once errors have been classified 
according to a typology, prescriptions for the correction of these types of 
errors can be incorporated into subsequent revisions of the programs. 

Clinchy and Rosenthal have proposed a classification scheme for learner 
errors.^ According to Clinchy and Rosenthal, three categories of errors 
exist: intake errors, organizational errors and executive errors. These 
three types of errors can be defined briefly as follows. Intake errors refer 
to "errors in the (learner's) conception of the goals and the data of the 
task." Organizational errors refer to errors made in the "selection of a method 
for manipulating the available information in order to reach a solution."-^ 
Finally, executive errors refer to errors in the "actual performance of the 
operations demanded by the program."'^ 

The purpose of the present study was to examine the feasibility of using 
Clinchy and Rosenthal's error classification scheme for the improvement of 
instructional programs. More specifically, the study was designed to answer 
two questions. 

1. Is the error classification scheme useful in differentiating the 
effectiveness of instructional programs? 

2. If the error classification scheme is useful in the above context, 
what prescriptions for program alteration can be made based on the types of 
errors made by learners in the programs? 



M_ethod 

Sample 



Two elementary schools, employing -.wo different instructional pro- 
irams. were us^d in the study. Both schools were located in the same 
school district and both schools contained students in grades one through 



six. 



The first school (School A) contained 135 boys and 141 girls. Eighteen 
per cent of the students received free or reduced lunches. Approximately 
fifteen per cent of the parents were classified as being in professional 
occupations. Virtually all of the students were Caucasian. 

The second school (School B) contained 239 boys and 220 -girls. Thirteen 
per cent of the students received free or reduced lunches. Approximately 
twenty-two per cent of the parents were classified in the professional 
occupational category. Again, virtually all students were Caucasian. 

Since two different schools were used in the study, an investigation 
of the initial cognitive differences of the students was conducted. Raw 
scores on the Metropolitan Readiness Test (MKT) administered at the beginning 
of first grade were available for students in grades two through five. 
Stainine scores on the Otis-Lennon Mental Abilities Test (OLMAT) administered 
in grade four were obtained for the grade six students.^ 

The mean scores of the students in the two schools at each grade level 
were computed and the differences compared using a one-way analysis of 
variance. Statistically significant differences favoring School B were found 
at grades three and six. Differences at grades two, four, and five also 
favored School B but were not statistically significant. 

Based on these findings, a decision was made to form matched pairs 
of students at each grade level, the matching being made on the student 
scores on the MKT or the OLMAT. ^ The >natching process produced 22 pairs of 
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students in the second grade, 21 pairs in the third grade, 22 pairi? in the 
fourth grade, 22 pairs in the fifth grade, and 23 pairs in the sixth grade. 
The means and standard deviations of the two groups of students at each 
grade on either the MRT or OLMAT were virtually identical. These luatche-i 
pairs formed the sample used in the study. 
Inst ructional. Programs 

School A teachers implemented an elementary mathematics program based 
on Block's lU'scr ini ion ;■, nuip-basoil mastery jLMniiiiK.'' Tlu. ScIuh> 1 
A teachers identified instructional objectives, designed original and 
supplementary instructional activities; and materials for each objective, 
constructed iorn,,.Live tests, and designed a system for recording and 
reporting individual pupil progress with respect to the objectives. The 
teachers began each instructional unit by informing the students of the 
objectives to be learned. The teachers then presented the original 
instructional materials and activities relevant to the particular subset 
of objectives to the entire class of students. Through the use of informal 
in-class questioning and worksheets as well as more formal tests, student 
progress vis a vis the objectives was periodically checked and recorded. 
Errors, once identified, were remediated through the use of large^group 
and small-group re-teaching (or corrective) sessions. After every student 
in the class had demonstrated a prosot level of "mastery" on either the 
unit test or a parallel form of the unit test, the entire class moved to the 
next instructional unit. 

School B teachers employed an instructional program believed to be 
representative of the conventional instructional programs in elementary schools, 
More specifically, the instructional program in School B was lacking in expli- 
city stated objectives, formative tests, re-teaching or corrective activities 
and materials, and a whole-class mastery performance standard. The study 
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lasted one academic year. 
Instrumentation 

Tests appropriate to each Rrade level were constructed for the purpose of 
assessing the effectiveness of the two instructional programs. Since pro.,. ,un 
effectiveness was defined in terms of student retention (rather than immediate 
achievement) the tests were dcsly.ncd Lo I,.. ,uiiu : n i sLo ro J oarlv in S..pt eirluT . 
The majority of the items on the test were constructed to represent the instruc- 
tional objectives of the instructional program operating in School A. Addi- 
tional items were written to parallel items on the California Achievement Test. 
Since the majority of the items were based on the objectives of School A's 
instructional program, one might suspect that the test might be differentially 
valid for the two instructional programs. In order to avoid this problem, 
a visit was made to School B where the inital set of objectives were discussed 
with the teachers at each grade level. Objectives that were not taught by 
teachers at both schools were identified and eliminated from further considera- 
tion. Teachers also examined the items and indicated those items they did not 
believe measured student attainment of the stated objectives. Such items 
although few in number, were eliminated from the tests. 

The items were written to test students understanding of arithmetic 
concepts, ability to perform arithmetic computations, and ability to perform 
algorithmic operations. All items were written in the short-answer coiv.plc- 
tion format. The internal consistency reliabilities of the five tests ranged 
from 0.80 to 0.92. The median internal consistency coefficient was 0.87. In 
general, then the tests seemed to possess an adequate degree of content vnliditv 
and internal consistency reliability. 
Scoring 

Following administration of the tests, the student responses were categorized 
according to the type of error made, one type of error for each correct answer. 
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Three categories of errors were established based on Clinchy and Rosenthal's 
orror c I ass L f L ra l i cni sohoivo. 

Computational errors most closely resembled Clinchy and Rosenthal's 
category of executive errors. Computational errors refer to those errors which 
indicated that the student had performed the correct arithmetic operation 
in accurately . That is, the student used the appropriate algorithm in solving the 
problem but made a mistake on one or more of the steps. An example of a compu- 
tational error would be the following. A student sees the problem 14 + 17, 
adds, but arrives at the answer 41. 

What may be termed "algorithmic errors" most closely resembled Clinchy 
and Rosenthal's category of organizational errors. Algorithmic errors refer to 
those errors which indicated that the student made use of an incorrect operation 
on algorithm. For example, the student may have added when he should have 
subtracted. Or, a student may have failed to "invert and multiply" when con- 
fronted with a problem dealing with division of fractions. 

Errors of omission, or omissions, were thought to most closely resemble 
Clinchy and Rosenthal's category of input errors. The reasoning behind this 
ecuation of error types was that students who failed to conceptualize the 
"aoals and data of the task" would be likely net to attempt the task at all. 
Omissions, therefore, included those problems the student made no apparent 
attempt to answer. 

In sum, then, each student response to each item was classified as to the 
type of error made: computational, algorithmic or omission. Where no mistake was 
made, of course, no error category was indicated. 
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Results 

The first question was concerned with the utility of the error classi- 
fication system in differentiating the two instructional programs. Lilii^s 1 
through 3 display the means, standard deviations, and correlated t-tests 
for the three categories of errors made by students in the two programs. 



Insert Tables 1, 2, and 3 About Here 

An examination of the differences in ciMiipiitaL ioiuil errors inn ^ by 
the students in the two programs (see Table 1) indicates that Scnool B students 
made significantly fewer computational errors than Scliool A students in v.^'^hIo 
five. In the remaining grades, no significant differences in the number of 
computational errors were found to exist. 

Quite a different picture is presented with respect to algorithmic errors 
(see Table 2). With the exception of students in grade four, students in 
School A made significantly fewer algorithmic errors than students in School B. 
In grade lour the difference was not significant. 

Finally, Table 3 presents the results with respect to student omissions. 
In the first three grades, no significant differences existed in the number 
of items omitted by students in the two instructional programs. In grades 
five and six, students in School B omitted significantly fewer items than students 
in School A. 

In general, then, the error classification scheme appears to be a useful 
way of distinguishing the instructional effectiveness of the two programs. 
The major difference between the two programs is in the area of algorithmic 
errors with the difference favoring instructional program A. Few differences 
between the programs exist in the areas executive errors and omissions. What 
differences do exist, however, tend to favor instructional program B. 
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Discussion 



The second question underlying the present study was concerned with 
the extent to which differences in types of errors could be used to improve 
instructional programs. The search for an answer to this question began with 
an examination of what each type of error might mean about the nature of 
students' learning. The following speculations were offered. 

Computational errors generally indicate that while students know how 
to think about the problem, they either are careless in performing the algorithms 
or are unaware of one or more of the mathematics facts necessary to perform 
the algorithm. On the other hand, algorithmic errors suggest the students, 
for some reason or reasons, selected the wrong algorithm to use in attempting 
to answer the question or solve the proble;n. Finally, omission errors typically 
indicate that the scudents do not know how to think about the problem. In 
other words, they do not even know where to begin. 

Based on these possible meanings, different alterations can be recoinmended 
for the different instructional programs yielding different types of errors. 
The major instructional weakness in the School B program was in the area of 
algorithmic errors. Two suggestions can be offered for such programs. 

First, a more systematic approach to the teaching of algorithms is 
reconmiended. This approach would include 1) an explicit display or presenta- 
tion of the steps involved in the algorithm; 2) a discussion of each of the 
steps, emphasizing the learner's understanding of each step; 3) a demonstrated 
application of the algorithm to a particular problem, complete with "answers" 
that follow from the successful application of each step; and 4) practice 
exercises which allow the student to apply the algorithm in a variety of 
situations and which require the student to write the partial answers after 
performing each step. The emphasis on the understanding of each step and the 
performance of the algorithm in a variety of situations would help the 



learners to see the limitations of the algorithm in solving problems. This 
may help the students to avoid over-application of the algorithm. 

Second, since algorithmic errors include errors in which the student 
adds instead of subtracts and divides instead of multiplies, it would be 
wise for the unit tests in such programs to be somewaht cumulative in nature. 
That is, unit tests could present addition and subtraction exercises, and 
multiplication and division exercises, m close proximity to each other. This 
will allow students practice in discriminating the algorithms that are neces.a 
to order to solve the problems correctly. 

In contrast to programs in which algorithimic errors aie Lhe major 
problem are those programs in which computational errorr constitute the 
major problem. In the present study, this problem see;,.s apparent only in the 
fifth grade program in School A. Whereas algorithmic ..M-,-ors soon: Lo n.,uiro 
alterations in the actual teaching and testing of the material or skill 
(as indicated above) , computational errors seem to require alterations in the 
amount or typ. of practice provided to the learners. Thus, instructional 
programs in which executive erron;occur frequently should provide the learners 
with additional practice exercises or different types of practice exercises. 
In this regard, students should check their answers to the exercises (or 
have their answers checked) frequently so that they do not overlearn incorrect 
computations . 

Instructional programs plagued by omissions would seem to benefit from 
a more systematic approach to the teaching of algorithms. The emphasis in 
this teaching should be on the relationship between problem situatiouc and 
appropriate algorithms rather than on the application of the algorithmic per su 

In conclusion, the use of evaluation as an aid in the improvement of 
instructional programs seems promising if. and only if. attention is focused 
on the types of errors made by the students on the tests. The error classi- 
fication scheme suggested by Clinchy and R-.senthal seems extremely 
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useful in this regard. Applications of the error classification scheme can 
aid in the identification of differences in the effectiveness of instructional 
programs which can, in turn, be used to suggest specific changes which can 
be made in order to improve the programs. 
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Table 1 



Means, Standard Deviations, Sample Sizes and Correlated t-tests for 
Number of Computational Errors Made by Students in the Two Programs 

in Grades 1 Through 5 



Grade Level 
2 

3 

4 

5 

6 



n 

22 

21 
22 
22 
23 



School A 



Mean 
3.1 

7.6 

5.4 

6.8 

4.3 



S.D. 
2.4 

3.7 

3.3 

2.8 

2.. 



School B 
Mean S.D. 



3.3 
6.4 
4.6 
4.8 
5.0 



4.1 
3.5 
2.4 
2.8 
3.3 



Correlated 
t-test 

0.0 

1.2 

0.7 

5.9* 

0.6 



Note The following convention for level of significance is used throughout 

this paper: .05 level = * and .01 level = ** 



Table 2 

Means, Standard Deviations, Sample Sizes and Correlated t-tests for 
Number of Algorithmic Errors Made by Students in the Two Programs 

in Grades 1 Through 5 



Grade Level 


n 


School A 


School B 


Correlated 
t-test 


Mean 


S.D. 


Mean 


S.D. 


2 


22 


0.2 


0.6 


3.4 


7-7 


3.8* 




21 


0.0 


0.0 


3.0 


6.1 j 


5.1* 


A 


22 


3.4 


2.7 


3.6 


4.5 i 


0.0 


5 


22 


5.3 


4.4 


13.1 


9.8 


11.7* 


6 


23 


10.3 


8.3 


17.0 


11.3 


5.2* 



Table 3 

Means, Standard Deviations, Sample Sizes and Correlated t-tests for Number 
of Omissions by Students in the Two Programs in Grades 1 Through 5 



Grade Level 


n 


School A 


School 


B 


Correlated 
t-test 


Mean 


S.D. 


Mean 


S.D. 


2 i 


22 


0.6 


2.6 


0.6 


2.8 


0.0 


3 


21 


1.0 


2.1 


0.2 


0.4 


2.9 


4 


22 


0.6 


1.7 


0.2 


0.9 


0.8 


5 


22 


4.0 


5.0 


1.5 


3.0 


3.9* 


6 


23 


12.3 


10.8 


6.3 


7.3 


4.9* 
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